Provincial  Report 


English  30 
Grade  12  Diploma  Examination 


April  1984 


/ 


Student  Evaluation 


/Liberia 

EDUCATION 


Distribution: 


Officials  of  Alberta  Education 
School  Principals 
Superintendents  of  Schools 
The  Alberta  Teachers'  Association 
Alberta  School  Trustees'  Association 
General  Public  Upon  Request 


TABLE  OF  CONTENTS 


PAGE 

PREFACE  iii 

ACKNOWLEDGMENTS  iv 

CHAPTER  1:  THE  GRADE  12  DIPLOMA  EXAMINATIONS  PROGRAM  1 

Introduction  1 

General  High  School  Diploma  1 

Advanced  High  School  Diploma  1 

Awarding  of  Course  Credits  1 

Transitional  Provisions  2 

Award  of  Excellence  2 

CHAPTER  2:  EXAMINATION  DESIGN,  DEVELOPMENT,  AND  DESCRIPTION  3 

Examination  Design  and  Development  3 

Examination  Description  4 

Part  A:  Written  Response  4 

Reporting  Categories  for  Part  A:  Written  Response  4 

Part  B:  Reading  (Multiple  Choice)  7 

Reporting  Categories  for  Part  B:  Reading  (Multiple  Choice)  7 

Taxonomic  Levels  for  Part  B:  Reading  (Multiple  Choice)  7 

CHAPTER  3:  ADMINISTRATION  OF  THE  EXAMINATION  11 

CHAPTER  4:  SCORING  OF  PART  A:  WRITTEN  RESPONSE  13 

Organization  of  Markers  13 

Training  13 

Marking  13 

Reliability  Reviews  14 

Scale-by-Scale  Assignment  of  Final  Mark  15 

Handling  of  Discrepant  Papers  15 

CHAPTER  5:  RESULTS  AND  OBSERVATIONS  17 

Total  Examination  Results  17 

Standard-Setting  17 

Relationship  Between  Examination  Marks  and  School -Awarded  Marks  18 
Results  for  Part  A:  Written  Response  19 

Section  I:  Critical  Response  to  Literature  19 

Section  II:  Personal  Response  to  Literature  20 

Section  III:  Major  Assignment  21 

Results  for  Part  B:  Reading  (Multiple  Choice)  21 

Discussion  of  Selected  Items  24 

Summary  of  Observations  27 

Total  Examination  27 

Part  A:  Written  Response  27 

Part  B:  Reading  (Multiple  Choice)  27 

APPENDIX  A:  ENGLISH  30  WRITTEN-RESPONSE  SCORING  GUIDES  29 

APPENDIX  B:  ENGLISH  30  SCORING  SHEET  33 

APPENDIX  C:  INDIVIDUAL  MARKER  PERFORMANCE  REPORT  34 


1 


LIST  OF  TABLES 


TABLE  PAGE 

1 English  30  Diploma  Examination  Blueprint  Part  A; 

Written  Response  6 

2 Distribution  of  Reading  Selection  Types  7 

3 English  30  Diploma  Examination  Blueprint  Part  B: 

Reading  (Multiple  Choice)  9 

4 Total  Examination  Results  17 

5 Percentage  Distribution  of  A's,  B's,  C's,  and  F's  18 

6 Percentage  Distribution  of  Scores  on  Section  I 19 

7 Percentage  Distribution  of  Scores  on  Section  II  20 

8 Percentage  Distribution  of  Scores  on  Section  III  21 

9 Results  for  Part  B:  Reading  (Multiple  Choice) 

(Raw  Scores)  22 

10  Item  Response  Frequencies  23 


- ii  - 


PREFACE 


This  report  presents  the  provincial  results  of  the  English  30  Diploma 
Examination  administered  on  January  17  (Part  A)  and  27  (Part  B),  1984.  During 
this  first  administration,  the  English  30  Diploma  Examination  was  written  by 
7879  students.  This  report  is  intended  to  provide  more  information  about  the 
examination  development  process,  the  examination  itself,  and  the  examination 
results. 
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CHAPTER  1 


THE  GRADE  12  DIPLOMA  EXAMINATIONS  PROGRAM 


Introduction 

All  Grade  12  students  in  Alberta  are  now  required  to  write  at  least  ONE 
diploma  examination  in  order  to  receive  a high  school  diploma.  The  Grade  12 
Diploma  Examinations  Program,  which  is  an  integral  part  of  the  high  school 
diploma  requirements,  is  intended  to  develop  and  maintain  excellence  in 
educational  standards  through  certification  of  academic  achievement. 

The  Diploma  Examinations  Program  consists  of  course-specific  examinations  that 
are  based  on  the  prescribed  Program  of  Studies  for  Senior  High  Schools  for 
the  following  Grade  12  (30-level)  courses:  English  30,  English  33,  Social 

Studies  30,  Mathematics  30,  Biology  30,  Chemistry  30,  and  Physics  30. 

Alberta  Education  issues  two  distinct  high  school  diplomas:  the  General  High 

School  Diploma  and  the  Advanced  High  School  Diploma. 

General  High  School  Diploma 

To  earn  a General  High  School  Diploma,  a student  must  obtain  course  credit  in 
either  English  30  or  English  33,  and  obtain  100  credits  distributed  over 
courses  as  specified  in  the  Junior-Senior  High  School  Handbook.  Some 
students  who  are  working  toward  the  general  diploma  may  wish  to  obtain  credits 
in  other  diploma  examination  courses  (i.e..  Social  Studies  30,  Mathematics  30, 
Biology  30,  Chemistry  30,  and  Physics  30).  To  obtain  credits  in  these 
courses,  a student  must  also  write  the  appropriate  diploma  examination 
regardless  of  which  type  of  diploma  he  wishes  to  receive. 

Advanced  High  School  Diploma 

The  Advanced  High  School  Diploma  represents  achievement  in  an  academic  program 
that  includes  language  arts  (English),  social  studies,  mathematics,  and 
science.  To  earn  an  Advanced  High  School  Diploma,  a student  must  satisfy  the 
current  course  and  credit  requirements  for  a General  High  School  Diploma  and 
obtain  course  credit  in  English  30,  Social  Studies  30,  Mathematics  30,  and  ONE 
of  Biology  30,  Chemistry  30,  or  Physics  30. 

Awarding  of  Course  Credits 

To  obtain  credit  in  any  Grade  12  (30-level)  course,  a student  must  earn  a 
final  mark  of  50%  or  better.  To  obtain  credit  in  a Grade  12  (30-level) 
diploma  examination  course,  a student  must  write  the  appropriate  diploma 
examination  and  attain  a final  blended  mark  of  50%  or  better.  The  "final 
blended  mark"  is  made  up  of  50%  of  the  mark  awarded  by  the  school  and  50%  of 
the  diploma  examination  mark.  For  example,  a student  taking  English  30  might 
have  a mark  of  45%  from  his  school  and  a mark  of  57%  on  the  diploma 
examination.  This  student’s  final  mark  is  the  average  of  the  two  marks,  or 
51%.  He  has  therefore  earned  credits  in  English  30  because  his  final  mark  is 
over  50%. 
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Transitional  Provisions 


During  the  1983-84  school  year,  Alberta  Education  will  recognize  all  course 
credits  earned  prior  to  September  1,  1983,  for  the  purpose  of  awarding  the 
General  High  School  Diploma. 

Students  who  have  completed  partial  requirements  for  the  Advanced  High  School 
Diploma  prior  to  September  1,  1983,  and  who  are  enrolled  in  Grade  12  courses 
during  the  1983-84  school  year,  may  apply  any  of  the  previously  completed 
required  diploma  examination  subjects  toward  a diploma,  provided  they  have 
earned  a final  course  mark  of  50%  or  better  in  each  subject. 

Award  of  Excellence 

When  a candidate  for  an  Advanced  High  School  Diploma  obtains  a final  average 
of  80%  or  higher  on  the  four  required  diploma  examination  courses  with  not 
less  than  65%  in  any  one  of  these  four  required  courses,  he  receives  an  Award 
of  Excellence.  This  Award  of  Excellence  is  noted  on  the  student's  Advanced 
High  School  Diploma. 

When  a student  writes  two  or  three  of  the  diploma  examinations  in  the  sciences 
(i.e..  Biology  30,  Chemistry  30,  and  Physics  30),  the  highest  of  these  final 
course  marks  is  used  for  diploma  purposes  and  in  the  calculation  of  the 
average  for  the  Award  of  Excellence. 
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CHAPTER  2 


EXAMINATION  DESIGN,  DEVELOPMENT,  AND  DESCRIPTION 


Examination  Design  and  Development 

There  were  four  stages  in  the  development  of  the  English  30  Diploma 
Examination:  preparation  of  curriculum  and  examination  specifications; 

examination  design,  development,  and  field-testing  of  questions;  and  selection 
of  questions  for  the  final  examination. 

The  Curriculum  Branch  of  Alberta  Education  prepared  curriculum  specifications 
that  identified  the  major  content  areas,  the  specific  objectives  within  each 
area,  and  the  emphasis  each  is  to  receive  in  the  classroom.  The  curriculum 
specifications  were  distributed  to  all  school  jurisdictions  in  the  province  in 
a document  entitled  Diploma  Examinations : Curriculum  Specifications  for 

English  30  (July  1983).  The  Student  Evaluation  Branch  of  Alberta  Education 
selected,  from  the  prepared  curriculum  specifications,  a set  of  diploma 
examination  specifications  that  formed  the  basis  of  evaluation  given  the 
limitations  of  a paper-and-pencil  examination  design.  These  diploma 
examination  specifications  constitute  Section  C of  Curriculum  Specifications 
for  English  30, 

The  Student  Evaluation  Branch,  in  consultation  with  Language  Arts  supervisors 
and  consultants  and  with  classroom  teachers,  then  designed  the  English  30 
Diploma  Examination.  The  examination  design,  complete  with  blueprints, 
scoring  guides,  and  sample  questions,  was  distributed  to  all  school 
jurisdictions  in  the  province  in  Student  Evaluation  Branch  Bulletin:  Volume 

3,  Number  2:  Grade  12  Diploma  Examinations  Program:  English  30  (September 

1983). 

Under  the  supervision  of  the  Student  Evaluation  Branch,  examination  questions 
were  developed  by  English  30  teachers  from  all  parts  of  the  province,  field 
tested,  and  revised.  Questions  were  designed  to  reflect  the  reading  and 
writing  objectives  outlined  in  the  diploma  examination  specifications. 

The  final  examination  was  constructed  from  those  questions  that  best  reflected 
curricular  intent  and  examination  design.  It  was  reviewed  by  a committee  of 
classroom  teachers  who  estimated  difficulty  levels  for  each  question  and 
suggested  revisions.  The  target  mean  for  the  English  30  Diploma  Examination 
was  set  at  S5%,  The  final  draft  was  approved  by  an  Examination  Review 
Committee  consisting  of  representatives  from  several  stakeholder  groups  (two 
Grade  12  English  teachers  representing  The  Alberta  Teachers'  Association;  one 
member  each  from  the  Conference  of  Alberta  School  Superintendents,  the 
Universities  Co-ordinating  Council,  and  the  Council  of  Presidents,  Public 
Colleges  of  Alberta),  and  representatives  of  the  Curriculum  Branch,  the 
Student  Evaluation  Branch,  and  the  Regional  Offices  of  Education.  The 
Examination  Review  Committee  checked  the  draft  for  content  validity,  accuracy, 
and  technical  merit.  Changes  to  the  examination  recommended  by  the  committee 
were  made. 
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Examination  Description 


The  English  30  Diploma  Examination  consisted  of  two  parts:  Part  A:  Written 

Response,  worth  50%  of  the  total  diploma  examination  score,  and  Part  B: 

Reading  (Multiple  Choice),  also  worth  50%  of  the  total  score. 

Part  A;  Written  Response 

Part  A consisted  of  three  sections.  Section  I:  Critical  Response  to 

Literature  required  the  student  to  read  a fable  and  a poem  (which  served  as 
stimuli  for  all  three  sections),  to  choose  a pair  of  symbols  from  one  of  the 
readings,  to  explain  what  each  item  in  the  pair  symbolized,  and  to  explain  how 
those  symbolic  meanings  developed  a contrast  and  contributed  to  the  main  idea 
in  the  reading.  Section  I was  worth  15%  of  the  mark  awarded  for  Part  A,  or 
7.5%  of  the  total  mark  for  the  English  30  Diploma  Examination.  Section  II: 
Personal  Response  to  Literature  required  the  student  to  examine  the  approaches 
to  life  adopted  by  the  characters  in  the  fable  and  the  poem,  to  choose  the 
approach  he  or  she  found  more  appealing,  and  to  explain  why  it  was  more 
appealing.  Section  II  was  worth  15%  of  the  mark  awarded  for  Part  A,  or  7.5% 
of  the  total  mark  for  the  examination.  Section  III:  Major  Assignment 

required  the  student  to  choose,  from  the  literature  studied,  two  characters 
who  are  similar  to  any  two  of  the  characters  from  the  fable  and  the  poem,  to 
compare  the  ways  in  which  the  two  characters  chosen  from  literature  approach 
life,  to  examine  the  motiviation  that  prompts  the  characters  to  approach  life 
as  they  do,  to  describe  the  consequences  of  the  choices  each  character  makes, 
and  to  express  an  opinion  about  those  choices.  The  major  assignment  was  worth 
70%  of  the  mark  awarded  for  Part  A,  or  35%  of  the  total  mark  for  the 
examination. 

Students  were  allowed  to  use  a dictionary  and  a thesaurus.  In  each  section, 
space  was  provided  for  planning  and  drafting  and  for  revised  work.  Students 
were  allotted  two  and  one-half  hours  to  write  Part  A. 

Reporting  Categories  for  Part  A:  Written  Response 

To  provide  information  about  student  writing  that  is  meaningful,  it  is 
necessary  to  examine  students'  responses  in  terms  of  writing  components  or 
factors  that  can  be  used  as  reporting  categories.  Factors  evaluated  in 
Sections  I and  II  were:  Thought  and  Detail  — the  content  present  in  the 

student's  writing,  the  concepts,  ideas,  and  literary  interpretations  presented 
and  the  specific  detail  selected  to  support  these  ideas;  and  Writing  Skills  — 
the  accuracy  and  effectiveness  with  which  the  student  used  diction,  sentence 
structure,  and  the  conventions  of  writing  (i.e.,  proper  spelling,  punctuation, 
and  grammar).  Section  III  was  assessed  according  to  five  reporting 
categories:  Thought  and  Detail  — the  content  present  in  the  student's 

writing,  the  ideas,  details,  literary  interpretation,  and  the  supporting 
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details  from  literature;  Organization  — the  coherent  development  of  a unified 
theme  or  thesis  through  an  appropriate  method  of  organization;  Matters  of 
Choice  — the  style  of  writing  used  by  the  student,  choices  of  words  and 
syntactic  structures  used  to  give  emphasis  and  effectiveness  to  writing;  and 
Matters  of  Convention  — the  mechanics  of  writing  and  the  conventions  of 
writing  (i.e.,  proper  spelling,  punctuation,  and  grammar).  The  design  of  Part 
A is  given  in  Table  1,  page  6. 
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Table  1 

English  30  Diploma  Examination  Blueprint 
Part  A:  Written  Response 
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Part  B:  Reading  (Multiple  Choice) 


Part  B:  Reading  (Multiple  Choice)  consisted  of  80  items  based  on  10  reading 

passages.  Information  on  the  numbers  and  types  of  reading  selections  on  Part 
B is  presented  in  Table  2. 


Table  2 

Distribution  of  Reading  Selection  Types 


Reading  Selection 
Type 

Number  of  Selections 

Number  of 
Items 

Modern  Drama 

1 

8 

Fiction 

2 

18 

Non-fiction 

3 

21 

Poetry 

3 

21 

Shakespearean  Drama 

1 

12 

Total 

10 

80 

Students  were  allotted  two  hours  to  write  Part  B.  Use  of  a dictionary  and 
thesaurus  was  not  allowed. 

Reporting  Categories  for  Part  B;  Reading  (Multiple  Choice) 

Items  were  grouped  into  three  reporting  categories  or  subtests:  Meanings 

(39  items).  Critical  Response  (16  items),  and  Human  Experience  and  Values  (25 
items).  Each  reporting  category  required  a minimum  of  six  items  to  achieve 
statistical  reliability.  The  reporting  categories  used  in  Part  B and  the 
proportion  of  the  total  examination  mark  assigned  to  each  reporting  category 
or  subtest  is  given  in  Table  3,  page  9. 

Taxonomic  Levels  for  Part  B:  Reading  (Multiple  Choice) 

A further  design  consideration  affecting  the  development  of  Part  B:  Reading 

(Multiple  Choice)  was  taxonomic  level.  Items  were  classified  according  to 
three  taxonomic  levels:  Literal  Understanding  (13  items).  Inference  and 

Application  (49  items),  and  Evaluation  (18  items).  By  considering  taxonomic 
level  when  developing  an  examination,  the  Student  Evaluation  Branch  attempts 
to  ensure  that  a variety  of  mental  activities  will  be  used  by  students  as  they 
write  the  examination.  Items  listed  under  Literal  Understanding  are  expected 
to  be  answered  using  skills  of  recognition  and  recall;  those  listed  under 
Inference  and  Application  are  expected  to  elicite  skills  of  analysis, 
interpretation,  and  application;  and  items  listed  under  Evaluation  are 
designed  to  draw  forth  judgmental  skills. 
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Since  there  is  no  way  of  knowing  for  certain  what  skills  have  been  applied  by 
a particular  student  or  group  of  students  when  answering  a particular 
examination  item,  results  have  not  been  reported  in  this  document  in  terms  of 
taxonomic  level.  Nonetheless,  taxonomic  level  was  an  important  design 
consideration  in  the  development  of  the  English  30  Diploma  Examination. 

The  classification  of  the  items  for  each  reporting  category  for  each  taxonomic 
level  is  shown  in  Table  3.  In  addition,  the  proportion  of  the  total 
examination  mark  assigned  to  each  taxonomic  level  is  given. 
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The  examination  presents  a variety  of  selections  from  fiction,  non-fiction  (literary  and  functional),  poetry,  and  Shakespearean  and  modern 
drama. 
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CHAPTER  3 


ADMINISTRATION  OF  THE  EXAMINATION 


In  December  1983,  the  Student  Records  and  Computer  Services  Branch  of  Alberta 
Education  received  Confirmation  of  Registration  lists  from  principals  of  all 
high  schools  in  Alberta  with  students  registered  in  Grade  12.  Principals  were 
asked  to  verify  the  registration  of  students  in  diploma  examination  courses 
within  their  schools  and  to  indicate  any  changes  in  registration  that  might 
have  occurred.  These  verified  lists  allowed  Alberta  Education,  through  the 
Student  Evaluation  Branch,  to  estimate  numbers  of  examinations  required  for 
each  diploma  examination  course  for  each  school.  It  also  allowed  each  answer 
sheet  and  scoring  sheet  to  be  prepared  with  specific  student  and  school 
identification  information. 

Also  in  December  1983,  each  superintendent  and  high  school  principal  was  sent 
a copy  of  the  document  entitled  Guidelines  and  Procedures  for  Administering 
the  Grade  12  Diploma  Examinations : January  1984  Administration.  This 

document  included  explanations  of  the  duties  and  responsibilities  of  the 
various  parties  involved  in  administering  the  examinations  (the  school 
jurisdiction,  the  school  principal  or  Chief  Presiding  Examiner,  and  the 
Presiding  Examiner).  In  addition,  information  about  security  of  the 
examinations,  confidentiality  of  students’  answers,  exceptional  cases,  and 
examination  rules  was  included. 

After  the  examinations  had  been  printed,  they  were  placed  in  school  packages 
by  subject  according  to  the  Confirmation  of  Registration  list  that  had  been 
verified  by  the  principal.  Each  school  was  sent  an  extra  number  of 
examinations  — approximately  lOZ  more  than  the  list  indicated.*  This  allowed 
for  changes  in  registration  and  provided  for  mature  students  — persons  at 
least  19  years  old  as  of  September  1,  1983,  who  had  been  out  of  school  for  at 
least  eight  consecutive  months  after  their  18th  birthday.  Mature  students 
were  allowed  to  write  any  diploma  examination  for  100%  of  the  final  mark 
without  being  currently  enrolled  in  the  course. 

A package  of  verified  copies  of  the  diploma  examinations  was  also  prepared  for 
each  Chief  Presiding  Examiner  (high  school  principal).  In  addition.  Chief 
Presiding  Examiner's  kits  (one  per  school)  and  Presiding  Examiner's  kits 
(approximately  one  for  each  30  students  in  the  school)  were  prepared.  The 
Chief  Presiding  Examiner's  kit  and  the  Presiding  Examiner's  kits  contained 
specific  instructions  necessary  to  ensure  reliable  administration  of  the 
examinations.  These  kits  also  contained  a copy  of  the  Examination  Rules. 

All  of  this  material  — the  packages  of  examinations,  the  verified  copies,  the 
Chief  Presiding  Examiner's  kits,  and  Presiding  Examiner's  kits  — was  sent  to 
the  central  office  of  each  school  jurisdiction  in  Alberta  in  early  January 
1984  for  dissemination  to  schools. 


*A11  high  schools  with  students  registered  in  Grade  12  received  a minimum  of 
three  extra  copies  of  each  diploma  examination  to  be  kept  on  file  in  the 
school  as  information  for  teachers  and  students. 
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School  jurisdictions  were  responsible  for  delivering  examination  materials  to 
schools  before  the  administration  dates.  The  written-response  portion  (Part 
A)  of  the  English  30  Diploma  Examination  was  administered  on  January  17,  1984, 
and  Part  B:  Reading  (Multiple  Choice)  v/as  administered  on  January  27,  1984. 

School  jurisdictions  were  responsible  for  keeping  the  examinations  secure 
prior  to  administration  and  for  supervising  the  collection  of  examination 
materials  from  schools  after  administration.* 

Examinations  were  returned  to  Alberta  Education  in  the  following  manner. 

School  jurisdictions  in  Zones  1,  4,  5,  and  6 returned  materials  as  directed  by 
their  Regional  Offices  of  Education.  Jurisdictions  in  Zones  2 and  3 returned 
materials  directly  to  the  Student  Evaluation  Branch  receiving  area  at  the 
Legislature  Annex,  9718  - 107  Street,  Edmonton.  At  the  Legislature  Annex,  the 
examinations  from  each  school  were  checked  against  the  school's  List  of 
Candidates  and  prepared  for  marking. 


*Grade  12  diploma  examinations  written  by  students  enrolled  in  category  I 
private  schools  were  administered  through  their  Acting  Superintendents  — 
usually  Regional  Office  of  Education  personnel. 


CHAPTER  4 


SCORING  OF  PART  A:  WRITTEN  RESPONSE 


Organization  of  Markers 

A marking  centre  was  established  in  Edmonton  on  the  fourth  floor  of  the 
Legislature  Annex.  One  hundred  and  fifty  teachers  from  across  the  province  of 
Alberta  spent  approximately  one  week  (January  30,  1984,  to  February  3,  1984) 
marking  Part  A:  Written  Response.  To  qualify  for  marking,  each  teacher  had 

to  have  a valid  permanent  Alberta  teaching  certificate,  to  have  taught  English 
30  for  at  least  two  years,  and  to  be  currently  teaching  English  30. 

The  150  markers  were  divided  into  25  groups  to  facilitate  the  administration 
of  marking.  Each  group  was  headed  by  a group  leader,  who  was  responsible  for 
helping  to  train  the  group  of  markers  and  for  conducting  reliability  reviews. 
The  25  group  leaders  were  brought  to  the  marking  centre  for  an  initial  meeting 
with  Student  Evaluation  Branch  personnel  on  Saturday,  January  28,  1984,  to 
respond  to  the  scoring  guides  (see  Appendix  A,  p.  29)  that  had  been  prepared 
and  to  read,  discuss,  and  come  to  a consensus  about  the  quality  of  papers 
selected  for  training  purposes. 

Training 

On  Monday,  January  30,  1984,  the  150  markers  met  at  the  Legislature  Annex. 

The  first  morning  was  used  for  training  the  150  markers.  The  training  began 
with  the  markers  being  divided  into  three  large  groups  of  50.  Student 
Evaluation  Branch  personnel  led  each  of  the  groups  through  an  overview  of  the 
marker's  manual,  a copy  of  which  had  been  provided  for  each  of  the  markers. 

The  criterion-referenced  scoring  scales  for  each  of  the  three  writing 
assignments  were  presented  to  the  markers.  Markers  then  regrouped  into  their 
25  small  groups  to  read  and  discuss  eight  papers  that  had  been  selected  for 
training  purposes.  Group  leaders  led  the  discussion  and  later,  again  in  the 
three  large  groups,  reported  the  opinions  expressed  in  their  small  groups. 
Student  Evaluation  Branch  personnel  then  led  the  large  groups  through  the 
mechanics  of  the  marking  procedure,  the  paper  flow,  and  procedures  for  filling 
out  forms. 

Marking 

The  remainder  of  the  week  was  used  to  evaluate  student  papers  independently. 
Additional  marking  sessions  were  held  on  a voluntary  basis  on  Tuesday, 
Wednesday,  and  Thursday  evenings,  and  on  Saturday,  February  4,  1984.  On 
Monday,  Tuesday,  and  Wednesday  of  the  following  week,  a group  of  10  markers 
met  to  complete  the  marking  of  the  last  batch  of  papers. 

Group  leaders  participated  in  standard-setting  for  Part  B:  Reading  (Multiple 

Choice)  on  Monday  afternoon,  Tuesday  at  noon,  Wednesday  at  noon,  and  Friday 
morning. 
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Before  papers  were  made  available  to  the  markers,  student  indentif ication 
information  was  removed,  and  papers  were  placed  into  bundles  of  eight.  The 
bundles  were  grouped  into  four  batches.  Batches  of  papers  were  marked  and  the 
score  sheets  processed  one  batch  at  a time,  so  that  papers  that  had  received 
discrepant  scores  could  be  identified,  pulled,  and  given  a fourth  reading 
during  the  marking  session. 

Because  information  identifying  the  student  had  been  removed  before  the  papers 
were  bundled,  markers  had  no  way  of  knowing  the  identity  of  the  student  who 
wrote  the  paper,  or  the  student's  school.  Markers  were  instructed  to  refer  to 
their  group  leader  any  paper  in  which  personal  knowledge  of  the  student  was 
suspected. 

All  papers  were  given  three  independent  markings.  A marker  would  take  a 
bundle  of  papers  from  a table  labelled  "Unmarked  Papers."  He  would  check  the 
back  cover  of  each  paper  in  the  bundle  to  make  certain  he  had  not  yet  read  any 
of  the  papers.  (Each  marker  was  assigned  a number  that  he  entered  on  the  back 
of  each  of  the  papers  he  marked. ) He  would  then  return  to  his  group  and 
independently  mark  each  of  the  papers  in  the  bundle  by  applying  the 
criterion-referenced  scales.  The  marker  would  assign  a mark  of  1 (low)  to  5 
(high)  on  each  of  the  scales  appropriate  to  the  assignments.  All  of  this 
information  was  placed  by  the  marker  on  a machine-scorable  sheet.  (See 
Appendix  B,  page  33.)  The  marker  would  place  his  marker  number  on  the  back 
cover  of  each  paper  in  the  appropriate  box,  and  return  the  bundle  of  papers  to 
another  table  labelled  "Marked  Papers."  The  process  would  then  be  repeated. 

Reliability  Reviews 

Reliability  of  results  was  of  prime  concern  during  the  marking  sessions. 
Because  of  this,  reliability  review  sessions  were  scheduled  twice  daily  — at 
10:00  a.m.  and  at  2:00  p.m.  At  these  sessions,  all  markers  were  given 
identical  papers.  The  original  plan  called  for  two  papers  to  be  used  in  each 
reliability  review,  but  because  two  papers  could  not  be  discussed  in  the  time 
allotted,  from  Tuesday  on  only  one  paper  was  presented  at  each  review.  All 
markers  independently  read  and  assigned  marks  to  the  paper,  and  the  group 
leaders  recorded  their  marks.  Each  group  then  discussed  the  paper  scale-by- 
scale, with  the  discrepant  or  divergent  markers  on  a particular  scale  speaking 
first.  At  the  end  of  the  reliability  review,  group  leaders  forwarded  their 
recorded  marks  to  Student  Evaluation  Branch  personnel,  who  then  tallied  all 
marks  and  posted  the  distribution  of  marks  for  that  session.  This  information 
provided  useful  feedback  for  monitoring  the  application  of  the  scoring  guides 
by  the  markers.  Group  membership  changed  at  the  Monday  and  Tuesday 
reliability  reviews.  Wednesday  and  Thursday,  markers  remained  with  their 
group  leaders  for  the  morning  reliability  reviews  and  group  membership  changed 
for  the  afternoon  sessions. 

To  provide  markers  with  additional  information  on  reliability,  individual 
marker  performance  reports  were  prepared.  See  Appendix  C,  page  34,  for  a 
sample  of  the  performance  report. 
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Scale-by-Scale  Assignment  of  Final  Mark 


Once  the  papers  from  a batch  had  received  three  markings,  their  scoring  sheets 
(see  Appendix  B,  page  33)  were  scanned  and  the  results  determined  by  a 
computer.  The  computer  was  programmed  to  assign  the  median  mark  for  each 
scale  as  the  final  mark.  If,  for  example,  the  first  marker  had  awarded  a 
particular  paper  a 3 for  Thought  and  Detail  on  Section  I,  the  second  marker  a 
2,  and  the  third  a 4,  the  computer  would  have  assigned  that  paper  the  median 
mark  of  3 for  that  particular  scale.  In  this  case,  the  marks  given  by  the 
second  and  third  markers  (2  and  4)  would  have  been  considered  divergent,  since 
they  each  differed  from  the  median  mark  by  one  scale  point.  If,  on  the  other 
hand,  the  first  marker  had  awarded  a 4,  the  second  a 4,  and  the  third  a 2,  the 
computer  would  have  assigned  the  median  mark  of  4 for  the  scale.  In  this 
second  case,  the  marks  given  by  the  first  two  markers  would  have  been  in 
agreement  (both  4's),  but  the  mark  given  by  the  third  marker  (2)  would  have 
been  considered  discrepant,  since  it  differed  from  the  median  mark  by  more 
than  one  scale  point. 

Handling  of  Discrepant  Papers 

Discrepancy  on  one  scale  alone  was  not  enough  to  require  a fourth  marking  for 
any  paper.  If,  however,  a number  of  scales  on  a particular  paper  were 
discrepantly  marked  by  the  first  three  readers,  the  scale  or  scales  that  were 
discrepant  were  given  a fourth  marking.  Marks  assigned  during  the  fourth 
marking  had  to  fall  within  the  range  of  marks  that  had  been  established  by  the 
first  three  markers.  If,  for  example,  the  first  marker  had  awarded  the  paper 
a 1 on  a particular  scale,  the  second  marker  a 3,  and  the  third  marker  a 3, 
the  fourth  marker  could  award  the  paper  a 1,  a 2,  or  a 3 on  this  same  scale. 
The  fourth  marker  could  not  go  outside  of  the  set  range  of  marks  to  assign  the 
paper  a 0,  a 4,  or  a 5 on  that  scale. 
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CHAPTER  5 


RESULTS  AND  OBSERVATIONS 


Total  Examination  Results 

The  results  of  the  English  30  Diploma  Examination  are  summarized  in  tables  4 
and  5.  The  average  mark  for  all  students  for  the  total  examination  was 
61.4%.  Students  achieved  an  average  of  30.4  marks  out  of  50  on  Part  A: 
Written  Response,  and  an  average  of  31.0  marks  out  of  50  on  Part  B:  Reading 

(Multiple  Choice). 


Table  4 


Total 

Examination 

Results 

Mean 

Standard  Deviation 

Total  Examination 

61.4% 

11.9 

Part  A:  Written 

Response 

30.4 

7.2 

Part  B:  Reading 

(Multiple  Choice) 

31.0 

8.1 

Standard  test  statistics  indicate  that  the  multiple-choice  section  has  a 
satisfactory  level  of  reliability  for  a test  measuring  a broad  range  of 
concepts  and  skills.  The  KR-20  coefficient  for  the  multiple-choice  part  of 
the  examination  was  0.852. 

Standard-Setting 

Every  effort  was  made  to  design  an  English  30  diploma  examination  that  would 
be  a valid  and  reliable  measure  of  what  students  can  be  expected  to  know  as  a 
result  of  instruction  in  this  course.  A specific  standard  or  level  of 
expectation  inherent  in  the  examination  was  established  through  careful  test 
development  procedures.  To  ensure  that  each  form  of  examination  administered 
in  1984  will  be  parallel  to  each  other,  the  Student  Evaluation  Branch  has 
adopted  a process  of  standard-setting.  One  way  to  review  the  standards 
inherent  in  each  examination  is  to  involve  classroom  teachers  in  making 
judgments  about  the  difficulty  of  the  examination. 
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The  teachers  who  assisted  with  training  markers  and  who  served  as  group 
leaders  during  the  marking  of  the  written-response  portion  of  the  examination 
also  served  as  standard-setters.  They  reviewed  the  difficulty  level  of  each 
question  in  terms  of  a borderline  passing  student  (who  merits  50%),  a 
borderline  "B"  student  (who  merits  65%),  and  a borderline  "A"  student  (who 
merits  80%) . After  teachers  gave  their  initial  judgments  on  question 
difficulty,  they  were  given  information  about  the  actual  distribution  of  the 
students'  examination  marks.  They  were  then  given  the  opportunity  to  modify 
their  judgments.  In  English  30,  the  teachers  recommended  that  the  actual 
distribution  of  the  students'  examination  marks  were  appropriate  at  the  50% 
and  65%  borderlines.  Although  they  suggested  that  a slightly  larger  number  of 
students  should  receive  A's,  it  was  decided  to  accept  the  actual  distribution 
as  the  standard. 

Relationship  Between  Examination  Marks  and  School -Awarded  Marks 

The  averages  for  examination  marks,  school -awarded  marks,  and  final  blended 
marks  were  61.5,  63.9,  and  63.2  respectively.  The  percentages  of  students 
receiving  A's,  B's,  C's,  and  F's  is  shown  below  in  Table  5. 


Table  5 

Percentage  Distribution  of  A's,  B's,  C's,  and  F's 


Letter 

Grade 

Examination 

Mark 

School -Awarded 
Mark 

Final  Blended 
Mark* 

A (80-100%) 

6.4 

9.2 

5.8 

B (65-79%) 

32.8 

40.0 

38.0 

C (50-64%) 

49.2 

42.8 

50.3 

F (0-49%) 

11.6 

8.0 

5.9 

*These  final  blended  marks  include  the  marks  of  students  who  did  not  receive 
BOTH  a school -awarded  mark  and  an  examination  mark  for  various  reasons  (see 
pages  1 and  2 ) . 


The  correlation  between  examination  marks  and  school -awarded  marks  was  0.56, 
indicating  that  a relatively  weak  relationship  exists  between  school -awarded 
marks  and  examination  marks.  It  is  normal  to  expect  differences  between  the 
two  marks  because  the  examination  score  is  limited  to  reflecting  achievement 
only  in  those  objectives  in  English  30  that  can  be  examined  by  a paper-and- 
pencil  test,  while  the  school -awarded  mark  reflects  achievement  in  listening, 
speaking,  viewing,  and  all  other  objectives  of  the  course.  The  school -awarded 
mark  also  reflects  the  quality  of  work  completed  by  students. 
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Results  for  Part  A:  Written  Response 


In  tables  6,  7,  and  8,  the  results  for  Part  A:  Written  Response  are  given. 

The  three  tables  examine,  in  turn,  the  three  different  sections  of  Part  A. 


Section  I:  Critical  Response  to  Literature 

Table  6 

Percentage  Distribution  of  Scores  on  Section  I 


Score 

(Scale  Points) 

Reporting  Category 

Thought  and 
Detail 

Writing 

Skills 

5 (Exceptional) 

7.1 

4.7 

4 (Proficient) 

30.1 

33.2 

3 (Satisfactory) 

39.6 

49.7 

2 (Limited) 

17.1 

9.2 

1 (Poor) 

4.4 

1.4 

0 (Insufficient 
or  No  Response) 

1.7 

1.7 

Observations  regarding  the  results  for  Section  I follow:  for  Thought  and 

Detail,  76.8%  of  the  students  scored  Satisfactory  or  better  (received  scores 
of  3,  4,  or  5),  and  21.5%  of  the  students  scored  Limited  or  Poor  (received 
scores  of  2 or  1).  For  Writing  Skills,  87.6%  of  the  students  scored 
Satisfactory  or  better,  and  10.6%  scored  Limited  or  Poor.  A total  of  1.7%  of 
the  students  did  not  attempt  Section  1,  or  produced  compositions  that  were 
considered  to  be  insufficient  for  marking  purposes  or  were  off-topic. 
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Section  II:  Personal  Response  to  Literature 


Table  7 

Percentage  Distribution  of  Scores  on  Section  II 


Score 

(Scale  Points) 

Reporting  Category 

Thought  and 

Writing 

Detail 

Skills 

5 

(Exceptional ) 

5.0 

3.9 

4 

(Proficient ) 

24.6 

28.3 

3 

(Satisfactory) 

51.5 

53.3 

2 

(Limited) 

14.3 

10.9 

1 

(Poor) 

2.6 

1.7 

0 

(Insufficient) 

2.0 

2.0 

Observations  regarding  the  results  for  Section  II  follow:  for  Thought  and 

Detail,  81.1%  of  the  students  scored  Satisfactory  or  better  (received  scores 
of  3,  4,  or  5),  and  16.9%  of  the  students  scored  Limited  or  Poor  (received 
scores  of  2 or  1).  For  Writing  Skills,  85.5%  of  the  students  scored 
Satisfactory  or  better,  and  12.6%  scored  Limited  or  Poor.  Two  per  cent  of  the 
students  did  not  attempt  Section  II,  or  produced  compositions  that  were 
considered  to  be  insufficient  for  marking  purposes  or  were  off-topic. 
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Section  III:  Major  Assignment 


Table  8 

Percentage  Distribution  of  Scores  on  Section  III 


Reporting  Category 

Total 

Thought  and  Organization 

Matters  of 

Matters  of 

Impression 

Detail 

Choice 

Convention 

5 (Exceptional) 

2.5 

3.0 

3.5 

3.3 

4 

4 (Proficient) 

19.4 

19.1 

23.5 

22.2 

31 

3 (Satisfactory) 

52.5 

43.9 

54.1 

56.3 

48 

2 (Limited) 

21.6 

29.0 

16.2 

14.8 

12 

1 (Poor) 

2.4 

3.5 

1.1 

1.8 

2 

0 (Insufficient) 

1.5 

1.5 

1.5 

1.5 

1 

Observations  about  the  results  for  Section  III  follow:  for  Total  Impression,  74.4^ 

of  the  students  scored  Satisfactory  or  better  (received  scores  of  3,  4,  or  5),  and 
24.0%  scored  Limited  or  Poor  (received  a 2 or  1).  For  Thought  and  Detail,  66.0%  of 
the  students  scored  Satisfactory  or  better,  and  32.5%  scored  Limited  or  Poor.  For 
Organization,  81.1%  of  the  students  scored  Satisfactory  or  better,  and  17.3%  scored 
Limited  or  Poor.  For  Matters  of  Choice,  82.8%  of  the  students  scored  Satisfactory 
or  better,  and  16.6%  scored  Limited  or  Poor.  For  Matters  of  Convention,  84.3%  of 
the  students  scored  Satisfactory  or  better,  and  14.1%  scored  Limited  or  Poor.  A 
total  of  1.5%  of  the  students  did  not  attempt  Section  III,  or  produced  compositions 
that  were  considered  to  be  insufficient  for  marking  purposes  or  off-topic. 

Results  for  Part  B:  Reading  (Multiple  Choice) 

The  results  for  Part  B:  Reading  (Multiple  Choice)  by  reporting  category  are 

presented  in  Table  9.  It  is  important  to  note  that  one  item  (#52)  from  the  first 
reporting  category.  Meanings,  was  deleted  from  Part  B.  This  meant  that  the  50%  of 
the  total  mark  for  the  English  30  Diploma  Examination  assigned  to  Part  B was 
calculated  from  a possible  raw  score  of  79.  Discussion  of  item  52  and  of  other 
selected  items  follows  later  in  this  report. 
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Table  9 


Results  for  Part  B:  Reading  (Multiple  Choice) 

(Raw  Scores) 


Reporting  Category 

Mean 

Standard 

Deviation 

1. 

Meanings  (38  items) 

24.7 

5.1 

2. 

Critical  Response  (16  items) 

10.5 

2.5 

3. 

Human  Experience  and  Values  (25  items) 

13.8 

3.9 

Although  performance  in  the  different  reporting  categories  appears  to  show 
some  variation,  caution  is  advised  in  comparing  them.  The  sets  of  items  that 
make  up  each  reporting  category  were  not  selected  to  be  equal  in  average  level 
of  difficulty.  Thus,  differences  may  be  due  to  variations  in  item  difficulty 
rather  than  in  student  performance.  In  combination  with  jurisdiction  results, 
however,  the  norms  can  be  used  to  detect  patterns  of  relative  strength  or 
weakness  in  achievement  in  each  of  the  reporting  categories. 
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Item  response  frequencies  for  all  80  items  appearing  in  Part  B:  Reading  (Multiple 

Choice)  are  presented  in  Table  10. 

Table  10 

Item  Response  Frequencies 


Item 

Number  Key 

Percentage  of  All  Students 
Choosing  Alternative 

Item 

Number 

Key 

Percentage  of  All  Students 
Choosing  Alternative 

A 

B 

C 

D 

A 

B 

C 

D 

1 

D 

7 

6 

9 

78 

41 

B 

11 

70 

16 

4 

2 

D 

14 

14 

7 

64 

42 

A 

70 

9 

4 

16 

3 

B 

5 

78 

7 

11 

43 

D 

4 

10 

8 

78 

4 

D 

5 

48 

7 

41 

44 

A 

53 

23 

12 

12 

5 

A 

96 

2 

2 

1 

45 

C 

14 

9 

60 

18 

6 

B 

5 

62 

23 

10 

46 

D 

8 

17 

8 

67 

7 

A 

49 

9 

14 

28 

47 

D 

7 

16 

1 

76 

8 

B 

17 

69 

12 

3 

48 

A 

46 

9 

23 

22 

9 

A 

75 

6 

9 

10 

49 

C 

4 

4 

85 

8 

10 

B 

4 

78 

17 

1 

50 

c 

17 

42 

40 

1 

11 

B 

3 

87 

6 

5 

51 

c 

27 

9 

60 

4 

12 

C 

17 

4 

69 

10 

52* 

D 

48 

15 

6 

31 

13 

A 

60 

2 

32 

6 

53 

A 

49 

4 

43 

4 

14 

A 

64 

10 

10 

16 

54 

A 

63 

13 

4 

20 

15 

B 

4 

96 

0 

0 

55 

C 

5 

26 

54 

15 

16 

B 

12 

77 

5 

6 

56 

C 

16 

4 

55 
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5 

*Item 

52  was 

deleted 

from  the 

examination. 

This  meant 

that  the 

50%  of  the  total 

examination  mark  assigned  to  Part  B was  calculated  out  of  a possible  raw  score  of 
79. 

Totals  may  not  add  to  100  due  to  rounding. 
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Discussion  of  Selected  Items 


Item  1 appeared  with  a short  poem  by  Alden  Nowlan  entitled  "Warren  Pryor." 
1.  In  line  7 the  word  "passport"  suggests 

A.  ambition 

B.  pilgrimage 

C.  career 

D.  deliverance 

Table  10  shows  that  1%  of  the  students  chose  alternative  A as  the  correct 
answer,  6%  chose  alternative  B,  9^  chose  alternative  C,  and  78%  chose 
alternative  D,  which  was  the  keyed  response.  This  particular  item  was  not 
very  difficult,  and  78%  of  the  students  answered  it  correctly. 


Item  39  proved  to  be  extremely  difficult  for  the  students.  It  appeared  with 
an  essay  by  Erich  Fromm  entitled  "Is  Love  An  Art?" 

39.  The  writer's  MAIN  purpose  is  to 

A.  caution 

B.  persuade 

C . condemn 

D.  classify 

Although  item  39  proved  to  be  a difficult  item  (only  21%  of  the  students 
answered  it  correctly) , it  was  decided  that  it  should  remain  a part  of  the 
examination  because  it  addresses  key  curriculum  objectives  (see  concepts  3,  4, 
5,  9,  10)  and  has  a clearly  correct  answer  readily  confirmed  by  the  text  of 
the  passage.  Students  may  not  have  taken  the  time  to  re-read  the  passage  in 
order  to  arrive  at  the  answer,  even  though  re-reading  was  necessary. 

Item  52  also  proved  to  be  quite  difficult  — 31%  of  the  students  answered 
correctly.  In  addition,  two  answers  drew  more  or  less  equally.  Although 
consensus  was  that  the  keyed  response  is  the  correct  one,  it  was  decided  to 
delete  item  52  because  students  seemed  unable  to  differentiate  between  the 
keyed  response  D (31%)  and  alternative  A (48%).  Item  52  appeared  with  an 
excerpt  from  the  play  Overlaid , by  Robertson  Davies. 

52.  Pop's  predominant  tone  in  the  selection  is 


A.  bitter 

B.  sentimental 

C.  tragic 

D.  mocking 
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Item  53  was  also  reviewed  because  some  teachers  were  concerned  that  it  did  not 
have  a clearly  correct  answer.  However,  the  item  did  not  prove  to  be 
problematic  for  the  students.  The  item,  also  based  on  Overlaid r appeared  on 
the  examination  as  follows: 

53.  The  fact  that  Ethel  calls  her  son  "Lover"  indicates  that  she 

A.  has  a distorted  understanding  of  love 

B.  thinks  she  is  being  funny 

C.  spoils  him  by  catering  to  his  every  wish 

D.  is  aware  that  he  has  been  charming  the  local  girls 

The  keyed  answer,  alternative  A,  was  chosen  by  49%  of  the  students,  while  43% 
chose  alternative  C.  However,  the  test  score  mean  of  those  who  chose  A was 
51,  and  for  those  who  chose  C was  47,  which  indicated  that  the  item  was 
discriminating  adequately.  For  these  reasons  it  was  decided  to  retain  the 
item. 

Item  65  was  also  reviewed  because  of  its  difficulty  — 29%  of  the  students 
answered  correctly.  Item  65  appeared  with  an  excerpt  from  Henry  VI,  Part 
III,  by  William  Shakespeare. 

65.  The  "she"  referred  to  in  line  32,  "She  did  corrupt  frail  Mature  with 
some  bribe,"  is 

A.  Law 

B . Love 

C.  Gloucester's  lady 

D.  Gloucester's  mother 

However,  it  was  decided  that  the  item  has  a clearly  correct  answer  that  is 
justifiable  from  the  passage.  As  well,  the  task  presented  by  the  item 
involves  a specific  skill  and  understanding  that  is  essential  for  Grade  12 
students  to  have  mastered  (understanding  and  determining  pronoun 
antecedents).  The  item  tests  concepts  3 and  11  of  the  curriculum,  and 
represents  a problem  often  found  in  students'  writing  as  well  as  a problem 
experienced  by  students  in  reading.  For  these  reasons  it  was  decided  to 
retain  item  65. 

Item  74  was  reviewed  because  it  also  proved  to  be  difficult  — only  25%  of  the 
students  answered  it  correctly.  Item  74  appeared  with  an  excerpt  from  "The 
Stream  Runs  Fast,"  by  L.  Rasmussen  et  al. 

74.  The  characters  of  both  Mrs.  Claude  Mash  and  Mrs.  Mellie  McClung  are 

A.  resolute  and  shrewd 

B.  cynical  and  embittered 

C.  tactful  and  diplomatic 

D.  fanatical  and  emotional 

This  item  was  retained  because  it  has  a correct  answer  justifiable  from  the 
passage,  and  because,  although  difficult,  the  item  discriminated  well.  The 
test  mean  for  those  choosing  the  correct  answer  was  55%,  and  for  those 
choosing  the  next  most  popular  alternative,  the  test  mean  was  48%.  As  well, 
the  item  tests  skills  that  are  stressed  in  the  curriculum. 


- 25 


Item  77  was  also  based  on  "The  Stream  Runs  Fast." 

77.  The  MAIN  reason  that  Sir  Radmond  Roblin  wishes  to  remain  ignorant  of 
the  conditions  in  the  factories  is  that  he  is 

A.  committed  to  preserving  the  established  order  of  power 

B.  revolted  by  the  appearance  of  the  garment  workers 

C.  angered  by  the  Canadian  Women's  Press  Club 

D.  concerned  about  the  financial  implications  of  change 

Although  some  teachers  expressed  concern  about  this  item,  it  was  retained 
because  it  is  justifiable  curricularly , has  face  validity,  and  has  a clear 
right  answer  based  on  the  text.  It  also  falls  within  the  acceptable  range  of 
difficulty. 
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Summary  of  Observations 


Total  Examination 

Achievement  on  the  English  30  Diploma  Examination  was  generally  regarded  as 
acceptable  by  teachers  involved  in  the  test  development  and  marking 
processes.  The  actual  examination  mean  of  61.4^  is  fairly  close  to  the  target 
mean  of  65.0%.  In  addition,  the  fact  that  only  11.6%  of  the  students  writing 
the  examination  scored  less  than  50%  suggests  that  students  are  successfully 
learning  those  concepts  and  skills  covered  by  the  English  30  Diploma 
Examination.  It  should  be  noted,  however,  that  there  were  few  high  achievers 
on  this  particular  examination.  Only  6.4%  of  the  students  scored  80%  or 
better. 

Part  A:  Written  Response 

Overall  achievement  on  Part  A:  Written  Response  was  fairly  good.  On  Section 

I:  Critical  Response  to  Literature,  the  majority  of  students  (76.8%  to  87.6%) 

scored  Satisfactory  or  better  depending  upon  the  reporting  category  being 
examined.  On  Section  II:  Personal  Response  to  Literature,  the  majority  of 

students  (81.1%  to  85.5%)  again  scored  Satisfactory  or  better  depending  upon 
the  reporting  category  being  examined. 

On  Section  III:  Major  Assignment,  student  performance  was  poorer  than  on  the 

minor  assignments.  To  do  well,  students  had  to  have  a good  understanding  of 
the  literature  they  chose  to  discuss.  On  Total  Impression,  74.4%  of  the 
students  scored  Satisfactory  or  better.  The  category  in  which  students  scored 
lowest  was  Thought  and  Detail  — 66.0%  scored  Satisfactory  or  better. 
Achievement  in  the  remaining  categories  of  Organization,  Matters  of  Choice, 
and  Matters  of  Convention,  was,  however,  quite  acceptable,  with  81.1%  to  84.3% 
of  students  scoring  Satisfactory  or  better. 

Part  B:  Reading  (Multiple  Choice) 

Achievement  on  Part  B:  Reading  (Multiple  Choice)  was  adequate.  Depending 

upon  the  reporting  category  or  subtest  being  examined,  means  varied  from  55.1% 
to  65.4%.  The  overall  mean  on  Part  B was  48.9  raw  score  marks  out  of  79. 

This  indicates  an  acceptable  standard  of  achievement. 
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APPENDIX  A 

ENGLISH  30  WRITTEN-RESPONSE  SCORING  GUIDE 


Minor  Assignment 

Thought  and  Detail  (Curriculum  Concepts  1 to  13  inclusive) 

5 EXCEPTIONAL  writing  presents  mature  and  insightful  ideas  enriched  by- 
effective  details  that  are,  in  themselves,  significant  and  memorable. 
Literary  interpretations,  where  required,  are  perceptive  and  defensible. 
The  choice  of  details  from  the  selections  contributes  significantly  to  the 
development  of  ideas. 

4 PROFICIENT  writing  presents  insightful  ideas  supported  by  appropriate 
details  OR  conventional  ideas  supported  by  effective  details  that  are,  in 
themselves,  significant  and  memorable.  Literary  interpretations,  where 
required,  are  perceptive  and  defensible.  The  choice  of  details  from  the 
selections  contributes  to  the  development  of  ideas. 

3 SATISFACTORY  writing  presents  conventional  ideas  supported  by  appropriate 
details.  Literary  interpretations,  where  required,  are  defensible.  The 
choice  of  details  from  the  selections  supports  the  ideas. 

2 LIMITED  writing  presents  conventional  ideas  that  are  either  unsupported  or 
are  accompanied  by  inappropriate  details.  Literary  interpretations,  where 
required,  are  incomplete  or  superficial.  The  choice  of  details  from  the 
selections  does  not  support  the  ideas  coherently. 

1 POOR  writing  presents  scant  ideas  and  details  that  do  not  develop  the 
topic.  Literary  interpretations,  where  required,  are  not  defensible.  The 
choice  of  details  from  the  selections  obscures  the  ideas. 

0 INSUFFICIENT  writing  is  so  deficient  in  length  that  it  is  not  possible  to 
assess  thought  and  detail. 

Writing  Skills  (Curriculum  Concepts  2,  3,  4,  5) 

5 EXCEPTIONAL  writing  communicates  cogently  through  accurate  and  effective 
diction  and  sentence  structure.  Effective  use  of  convention  enhances  the 
communicative  power  of  the  composition. 

4 PROFICIENT  writing  communicates  capably  through  accurate  and  generally 
effective  diction  and  sentence  structure.  Errors  in  spelling, 
punctuatioT  and  grammar  do  not  reduce  the  communicative  power  of  the 
composition . 

3 SATISFACTORY  writing  communicates  clearly  through  generally  accurate  and 
generally  effective  diction  and  sentence  structure.  Errors  in  spelling, 
punctuation,  and  grammar  reduce  but  seldom  impede  the  communicative  power 
of  the  composition. 

2 LIMITED  writing  communicates  incompletely  through  generally  accurate  but 
frequently  ineffective  diction  and  sentence  structure.  Errors  in 
spelling,  punctuation,  and  grammar  often  impede  the  communicative  power  of 
the  composition. 

1 POOR  writing  communicates  unsuccessfully  through  frequently  inaccurate  and 
generally  ineffective  diction  and  sentence  structure.  Errors  in  spelling, 
punctuation,  and  grammar  severely  impede  the  communicative  power  of  the 
composition. 

0 INSUFFICIENT  writing  is  so  deficient  in  length  that  it  is  not  possible  to 
assess  the  writing  skills. 
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APPENDIX  A (CONTINUED) 


Major  Assignment 

Total  Impression  (Curriculum  Concepts  1 to  13  inclusive) 

5 EXCEPTIONAL  writing  presents  perceptive  ideas,  supporting  details,  words, 
and  structures  that  form  a coherent  and  unified  whole.  The  writer's 
purpose  and  intended  audience  are  unquestionably  evident  throughout  the 
work,  and  his  tone  is  both  apparent  and  always  appropriate.  This  writing 
communicates  clearly,  effectively,  and  with  vitality.  Such  writing 
demands  respect. 

4 PROFICIENT  writing  presents  appropriate  ideas,  supporting  details,  words, 
and  structures  that  form  a coherent  and  unified  whole.  The  writer's 
purpose  and  intended  audience  are  clear  throughout  the  work,  and  his  tone 
is  generally  apparent  and  appropriate.  This  writing  communicates  clearly 
and  effectively.  Such  writing  prompts  attention. 

3 SATISFACTORY  writing  presents  conventional  ideas,  supporting  details, 
words,  and  structures  that  form  a coherent  and  unified  whole.  The 
writer's  purpose  and  intended  audience  are  generally  clear  throughout  the 
work,  and  his  tone,  when  apparent,  is  appropriate.  This  writing 
communicates  clearly.  Such  writing  is  usually  accepted  as  adequate. 

2 LIMITED  writing  presents  casual  ideas,  supporting  details,  words,  and 

structures  that  lack  coherence  and  unity.  The  writer's  purpose,  intended 
audience,  and  tone  are  generally  unclear.  This  writing  communicates 
ineffectively  or  incompletely.  Such  writing  does  not  satisfy  the  reader. 

1 POOR  writing  lacks  a central  idea,  purpose,  or  focus.  Coherence  and  unity 
are  not  achieved.  The  writer's  purpose  and  intended  audience  are 
unclear.  The  tone  is  either  inappropriate  or  unclear.  This  writing 
impedes  communication.  Such  writing  frustrates  the  reader. 

0 INSUFFICIENT  writing  shows  that  the  assignment  has  not  been  attempted  or 
that  the  attempt  is  of  insufficient  length,  or  that  the  attempt  is 
completely  off-topic. 
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APPENDIX  A (CONTINUED) 


Thought  and  Detail  (Curriculum  Concepts  1-13  inclusive) 

5 EXCEPTIONAL  writing  presents  mature  and  insightful  ideas  enriched  by 
effective  details  that  are,  in  themselves,  significant  and  memorable. 
Literary  interpretations  are  perceptive  and  defensible.  The  choice  of 
details  from  literature  contributes  significantly  to  the  development  of 
ideas . 

4 PROFICIENT  writing  presents  insightful  ideas  supported  by  appropriate 
details  OR  conventional  ideas  supported  by  effective  details  that  are,  in 
themselves,  significant  and  memorable.  Literary  interpretations  are 
perceptive  and  defensible.  The. choice  of  details  from  literature 
contributes  significantly  to  the  development  of  ideas. 

3 SATISFACTORY  writing  presents  conventional  ideas  supported  by  appropriate 
details.  Literary  interpretations  are  defensible.  The  choice  of  details 
from  literature  supports  the  ideas. 

2 LIMITED  writing  presents  conventional  ideas  that  are  either  unsupported  or 
are  accompanied  by  inappropriate  details.  Literary  interpretations  are 
incomplete  or  superficial.  The  choice  of  details  from  literature  does  not 
support  the  ideas  coherently. 

1 POOR  writing  presents  scant  ideas  and  details  that  do  not  develop  the 
topic.  Literary  interpretations  are  not  defensible.  The  choice  of 
details  from  literature  obscures  the  ideas. 

0 INSUFFICIENT  writing  is  so  deficient  in  length  that  it  is  not  possible  to 
assess  thought  and  detail. 

Organization  (Curriculum  Concepts  2,  3,  4,  5) 

5 EXCEPTIONAL  writing  presents  a compelling  introduction,  successfully 
constructed  to  provoke  further  reading.  The  middle  is  crafted  so  that  the 
sentences  and  ideas  flow  smoothly  to  an  effective  conclusion. 

4 PROFICIENT  writing  presents  an  effective  introduction  that  attempts  to 
encourage  further  reading.  The  middle  is  developed  so  that  sentences  and 
ideas  flow  smoothly  to  an  appropriate  conclusion. 

3 SATISFACTORY  writing  presents  a focused  and  functional  introduction, 
middle,  and  conclusion.  Sentences  and  ideas  are  clearly  related  but 
coherence  occasionally  falters. 

2 LIMITED  writing  may  lack  an  introduction,  or  have  an  introduction  that  is 
not  functional.  The  conclusion,  if  attempted,  may  be  unsuccessful. 
Relationships  among  sentences  and  relationships  among  ideas  are  frequently 
unclear . 

1 P(X)R  writing  presents  neither  an  introduction  nor  a conclusion.  Sentences 
and  ideas  are  presented  in  no  particular  order. 

0 INSUFFICIENT  writing  is  so  deficient  in  length  that  it  is  not  possible  to 
assess  the  organization  of  the  material. 
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APPENDIX  A (CONTINUED) 


Matters  of  Choice  (Style)  (Curriculuin  Concept  3) 

5 EXCEPTIONAL  writing  presents  a selection  and  use  of  words  and  sentence 
structures  that  is  accurate,  effective,  and  often  polished.  Choices 
appear  to  have  been  made  deliberately  to  achieve  a particular  purpose. 

Tone  is  appropriate,  discernible,  and  controlled. 

4 PROFICIENT  writing  presents  a selection  and  use  of  words  and  sentence 
structures  that  is  accurate  and  generally  effective.  Choices  frequently 
appear  to  have  been  made  deliberately  to  achieve  a particular  purpose. 

Tone  is  appropriate,  discernible,  and  usually  controlled. 

3 SATISFACTORY  writing  presents  a selection  and  use  of  words  and  sentence 
structures  that  is  generally  accurate  and  generally  effective.  Choices 
occasionally  appear  to  have  been  made  deliberately  to  achieve  a particular 
purpose.  Tone,  when  discernible,  is  appropriate. 

2 LIMITED  writing  presents  a selection  and  use  of  words  and  sentence 
structures  that  is  frequently  inaccurate  and  ineffective.  Choices  seldom 
appear  to  have  been  made  deliberately  to  achieve  a particular  purpose. 

Tone  may  be  absent  or  inappropriate. 

1 POOR  writing  presents  a selection  and  use  of  words  and  sentence  structures 
that  is  frequently  inaccurate  and  ineffective. 

0 INSUFFICIENT  writing  is  so  deficient  in  length  that  it  is  not  possible  to 
assess  matters  of  choice. 

Matters  of  Convention  (Curriculum  Concept  3) 

5 EXCEPTIONAL  writing  is  essentially  free  from  errors  in  spelling, 
punctuation,  and  grammar.  The  effective  use  of  conventions  enhances  the 
communicative  power  of  the  composition. 

4 PROFICIENT  writing  is  essentially  free  from  errors  in  spelling, 
punctuation,  and  grammar.  Errors  that  are  present  do  not  reduce  the 
communicative  power  of  the  composition. 

3 SATISFACTORY  writing  has  occasional  errors  in  spelling,  punctuation,  and 
grammar.  These  errors  reduce  but  seldom  impede  the  communicative  power  of 
the  composition. 

2 LIMITED  writing  has  frequent  errors  in  spelling,  punctuation,  and 
grammar.  These  errors  often  impede  the  communicative  power  of  the 
composition. 

1 P(X)R  writing  has  errors  in  spelling,  punctuation,  and  grammar  that  are 
both  noticeable  and  jarring.  These  errors  severely  impede  the 
communicative  power  of  the  composition. 

0 INSUFFICIENT  writing  is  so  deficient  in  length  that  it  is  not  possible  to 
assess  matters  of  convention. 
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